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1. BACKGROUND 


The first known regulation to control a profession by specifying a professional license 
was established in 1140 A.D. when King Roger of Normandy decreed that all physicians present 
proof of competency before being allowed to practice medicine. After successfully completing 
all required academic work and a written comprehensive examination, the physician would be 


certified to provide medical services to the public. 


In the early 1760’s John Smeaton identified himself as the first “civil engineer” to 
distinguish himself from the “military” engineers and to clearly designate his target market. In 
1852, the United States began recognizing civil engineering through the establishment of the 
American Society of Civil Engineers (ASCE). At that time membership in ASCE was sufficient 
to be recognized as a professional engineer by the courts. During the 1870’s and 1880’s, a 
number of railroad bridge failures occurred, resulting in a tremendous loss of life. Many of these 
failures were consistent with the evolving nature of the engineering profession that included new 
materials, innovative construction systems and more complex load distributions associated with 
longer span bridges. The public attributed the accidents to incompetent and unqualified 


engineers. This led to the need for legal registration of the profession in 1897. 


In 1907, after much discussion, a panel of 11 ASCE members in Wyoming established 
the first licensing law governing the practice of engineering and surveying. After the bill was 
passed, written examinations began and within 2 years, 134 engineers were registered in 
Wyoming. Other states soon enacted their own legislation and by 1947, every state had 
legislation controlling the registration of engineers; however, licenses granted by one state were 
not recognized by another states. In 1920, the Council of State Boards for Engineering 
Examiners was established, and later renamed to the National Council of Engineering Examiners 
(NCEES) in 1967, in an attempt to promote more uniform licensing standards, national licensure, 
and reciprocity between the states. Although each State maintains specific individual licensing 
requirements, all states use the exam prepared by the NCEES to ensure each examinee is 


provided with the same opportunity to become licensed. 


The licensing requirements for Civil Engineering include documentation of relevant work 
experience, personal references, and a written exam. The original licensure exam involved a 4- 
hour written problem solving section, which allowed the examinee to receive partial credit. Due 
to the non-uniform and subjective in-state grading practices, the NCEES changed the exam 
format. The current NCEES exam format, which began in October 2001, for a general Civil 
Engineering exam consists of an 4-hour multiple choice “depth” exam concentrating on one of 
the 5 major topic areas: geotechnical, environmental, structural, transportation, water systems, 
following a 4-hour multiple choice “breadth” exam that covers all areas of Civil Engineering 


(20% of the questions from each area). 


According to data gathered from the NCEES an estimated 23,500 Civil Engineers register 
to take the Professional Engineer (P.E.) licensure exam each year. Approximately 400 of these 
examinees apply to take the exam in North Carolina. Study and preparation for the P.E. exam is 
a major concern for engineers seeking their license. Most P.E. candidates are practicing 
engineers that depend on a variety of study resources and refresher classes due to the “breadth” 


of the examination 


The Industrial Extension Service (IES) of North Carolina State University is a provider of 
a high quality P.E. refresher course which is taught in a traditional live classroom format. The 
IES is currently considering an expansion of the P.E. refresher course by means of utilizing a 
suitable distance learning method to reach a larger audience while maintaining a high quality of 


P.E. training offerings. 


In a survey done by the IES in early 2001, 31 of 32 engineers preparing for the exam 
purchased some sort of study material. Of these 32, 17 attended a live training course, but 28 
would consider a distance learning course. Additionally, 23 of these 28 would purchase a CD- 
ROM and textbook distance learning course if the price and quality was competitive to a live 


course. 
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2. PROBLEM STATEMENT 


Engineers seeking to obtain Professional Engineer licensure typically attend a formal 
classroom refresher course to help prepare for the written, qualification exam, especially those 
practicing abroad. Due to the flexibility to meet the constraints of practicing engineers, distance 
learning methods have become useful tools to assist in preparation for the Professional Engineer 
exam. Distance learning formats include video lectures, live online classes, and self-paced CD- 
ROMs. As a result of continually improving technology, distance learning methods have 
become more complete and affordable. The survey by IES also indicated that engineers would 


strongly consider distance learning methods to prepare for the P.E. exam. 


As a provider of a “traditional” P.E. review course, the IES requires a quick survey of 
different existing distance learning delivery methods for the P.E. review. Several currently 
available Civil Engineering P.E. refresher distance learning courses were examined and the one 
“best” alternative was selected for an in-depth analysis to enable IES to offer a P.E. review 
course via distance learning. The in-depth subject evaluation of the selected course was 
completed from the point of view of a “typical” engineer preparing for the exam to identify the 
advantages and disadvantages of the course and to assist IES in the development of a separate, 


independent distance-learning course. 


This report summarizes the various P.E. distance learning courses currently available, 
describes the selection criteria for evaluating one specific course, and discusses the advantages 
and disadvantages of the course. Recommendations are provided for the IES for possible 


implementation and further study. 


3. OBJECTIVES 


3.1 General 


Obtaining P.E. Licensure is a defining milestone in an engineer’s career. The traditional 
method of preparing for the written exam involves a self-directed or classroom refresher course. 
Self-directed studies lack structure and interaction. Traditional classroom courses lack flexibility 
to meet the working schedule of a typical engineer and have limited attendance due to 
constrained geographic location. With rapidly developing technology, several different distance 
learning courses become available which offers and attractive alternative to the “traditional” 


means of examination review. 


3.2 Project Objectives 


The objectives of this project are as follows: 

e Identify critical factors that would make distance learning courses a desirable 
alternative 

e Identify various representative distance learning P.E. review courses currently 
available 

e Select a “best” course alternative 

e Evaluate the selected course and identify advantages and disadvantages 

e Develop break even cost evaluation for product development 

e Provide feedback to the IES for future development of a distance learning 


course 


3.3 Limitations 


This report was completed with the following limitations: 
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e Completeness / Exhaustiveness: Not all available P.E. refresher courses were 
investigated. Only representative courses that were found via an internet search were 
considered for the initial sample of distance learning courses. 

e Objectiveness: The analysis of the evaluated course is subjective and represents the 
view of a practicing engineer preparing for the P.E. exam. This objectiveness also 
applies to the identification of relative weight factors and other non-scientific criteria 
used for evaluation purposes. 

e Subjective Ranking: All rankings were based on the perception and subjectivity of 
the evaluating engineer and were not a measure of the course content quality. 
Rankings were used to produce a relative measure of completeness, through a 


desirable product matrix, for future product development purposes. 
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4. METHODOLOGY 


4.1 General 


This study was conducted in three phases. Phase I consisted of identifying a sample of 
available distance learning courses and developing a product evaluation matrix that was based on 
the course description as advertised by the seller. Phase II involved selecting and performing a 
detailed evaluation of one course. Phase III consisted of writing a report with specific 


recommendations for development of a distance learning course by IES. 


All phases of this report were conducted by an engineer (“the evaluator’) that meets the 
profile of a typical examinee likely to use a distance learning course to prepare for the P.E. 
examination. The evaluator was a graduate student completing a Master’s Degree in Civil 
Engineering. The evaluator had previously obtained a Bachelor’s degree in Aerospace 
Engineering from a 4 year ABET accredited university. The student had met all experience 


requirements for the P.E. exam and had 6 months to prepare for the Civil Exam. 


4.2 Phase I: Course Identification 


Available distance learning refresher courses were identified through an extensive 
Internet keyword search. Additionally, the National Society of Professional Engineers (NSPE), 
the American Society of Civil Engineers (ASCE), and the National Council of Examiners for 
Engineering and Surveying (NCEES) were contacted to identify additional courses. The NCEES 
administers exams throughout the country but does not provide refresher courses. Contacts with 
NSPE and ASCE did not yield course offerings different from those already identified. A 
representative list of currently available P.E. Review courses is outlined in Appendix A, along 


with corresponding course format and contact information. 
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4.3 Phase II: Course Selection 


The primary types of distance learning course formats are classroom, textbook, video, 
and CD-ROM. Each of these formats can be presented in two ways: online and offline courses. 
Online courses involve the use of the internet to interact with others using one of the four 
methods. The reference material from an online textbook course could be downloaded from the 
internet. Offline courses concentrate on self-paced learning that is not associated with the 


internet. The advantages and disadvantages of each method and format are listed in Table 1. 


Table 1: Distance Learning Course Format Summary 


Face-to- Face Contact Scheduling Difficulties 
Real Time Feedback / Assistance Extensive Instructor Involvement 
Classroom | Group Discussion Easily Conducted Live Internet Course Expensive to 
(On-line & Administer 
Offin) [Lack of Flexibility in Attendance _| 
Review Course Lengths typically 
longer than other formats 


Good Reference to Take Into the P.E. No Student / Teacher Interaction 
Textbook Exam 


(On-line & ger Variety of Practice Problems No Feedback 
Offline) All Material is Self-Taught with No 
, Instruction 


Ability to See the Instructor on Pre- Minimal Feedback 
Recorded Lesson 


Minimal Instructor Involvement with Minimal Student / Teacher Interaction 
CD-ROM Course after Initial Development 


(On-line & | Printable Text Available for Reference Limited Variety of Problems 
Offline) during P.E. Exam 
igh Quality Visual Aids 
Lesson Topic Instruction Quick and 
Easy to Repeat 
Ability to See Instructor No Feedback 
Minimal Instructor Involvement with No Student / Teacher Interaction 
Cidee Course after Initial Development 
(Offline) No Written Text or Example Problems 
Provided 


Lo Review of Specific Topics are Slow to 
Repeat 
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The online, live classroom environment is most effective in allowing people to interact 
and discuss problems and solutions. This format is costly to produce over the internet and 
requires specific scheduling by students. The textbook is a very common method of review and 
provides a useful reference that can be brought into the exam. Use of a textbook is designed for 
self-paced learning without the benefit of personnel interaction. CD-ROM and video courses 
combine the advantages and disadvantages of classroom and textbook distance learning formats 
into a useful and effective course. The advantages of each method and delivery mechanism were 
integrated into a more formal evaluation procedure to select a “best” course to meet IES 


requirements. Full discussion of the evaluation procedure is provided in Chapter 5. 


4.4 Phase II: Course Evaluation 


A comparison chart was first developed to summarize the advantages and disadvantages 
of each module in the evaluated course and create a set of “ideal” course characteristics. Several 
references were used to identify key success factors that contribute to the development of an 
effective learning environment. A hypothetical “ideal” learning environment was developed by 
combining these key factors into a single comparison chart. The effectiveness of any training 
course is dependent on the individual needs of the student, and the proper implementation and 


combination of these keys factors should improve the learning capability of an average student. 


A value of 100% was assigned to each factor to establish a baseline value for comparison. 
Each key factor was assigned a weight factor of importance based on the similarities between the 
references and the level of importance as based on the evaluator’s judgement. These weight 
factors were used to determine an overall rating of the individual modules compared to a 
hypothetical “ideal” learning environment. The completed comparison chart is shown in 


Appendix B. 


Following the creation of the “ideal” course characteristics chart shown in Appendix B, 
the evaluator enrolled and completed individual sections of the course. Upon completion of each 


individual module, the positives and negatives of each section were compared with the ideal 


course characteristics in Appendix B, and a rating was assigned to each characteristic as 
compared to the pre-determined baseline. The individual module ratings were based on how 
effectively the module met or did not meet the selection criteria. After completing the matrix for 
each module, the individual ratings were multiplied by its weight factor and combined to 


determine and overall rating for the course. 


After evaluating each module and completing the pre and post-exams offered with the 
course, recommendations were developed which identified existing and additional topical 
coverage areas, time to cover each of the respective topic areas, presentation format, and types of 
example problems to present. In addition, modifications appropriate for development of a new 


course offering by IES, including the incorporation of new technologies, were recommended. 


5. COURSE SELECTION CRITERIA 


5.1 General 


Course selection criteria were established based on on-line marketing information 
provided by the product seller. A total of 7 common product features were identified and are 
summarized in Appendix C. Each feature was assigned a individual weight factor based on 
perceived student needs and the requirements set forth by the IES. The combination of all seven 
factors gave a total possible 100 points. A full discussion of each contributing factor is provided 


in sections 5.2 through 5.8. 
5.2 Course Format 


The Industrial Extension Service desires to develop a new course that captures the quality 
of their live course and instructors, yet is self-sustaining in the sense that it requires minimal 
post-development instructor involvement. Presentation format is a critical factor in determining 
which course to select. A live Internet course most closely represents a classroom environment 
and allows student interaction; however, it would require continuous instructor involvement and 
is more costly than other methods. There are numerous stand-alone textbooks available which 
have saturated the marketplace and made it difficult for a new book to have a competitive 


advantage over others. 


Both the on-line and offline CD-ROM and video based formats were the most applicable 
to this project. Since the Industrial Extension Service currently produces a video based refresher 
course which covers some of the major topics on the P.E., the on-line CD-ROM format was 
selected as the most appropriate due to its lower cost and flexible scheduling. Based on the 
importance of selecting the correct format, this factor was assigned a maximum weight factor of 


20 out of 100. 
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5.3 Availability of Instructor 


Inevitably questions arise from difficult and unclear concepts, or details that were not 
addressed in the lecture. The type of assistance and the availability of the instructor became the 
second factor of importance. Only classroom or video-based courses that provide a means of 
interaction between the students and instructor, through either email, telephone, or face-to-face 
contact, were considered acceptable courses. This factor was considered critical in the 
effectiveness of a distance learning course and was also assigned a maximum weight factor of 20 


out of 100. 


5.4 Class Duration 


Typically, individuals preparing for the P.E. exam through the use of a classroom based 
refresher course begin reviewing material a few months prior to the exam date. Self directed 
refresher courses are often completed in less time than a structured class. Designing a new 
structured course, but on a self-paced level, caused the course completion time to be the third 
factor of importance. The time required to complete any course is partially dependent on the 
individual and on the designed course length. Courses evaluated in this study ranged from 50 
hours of self-paced instruction to 6 months of interactive class work. Since the typical individual 
who will be completing the review course will be maintaining a full time job, ideally, the shorter 
amount of time required for review, the more likely the individual is to choose a specific course. 
With a wide range of available course durations, this factor was assigned a maximum weight 


factor of 15 out of 100. 
5.5 Instructor’s Knowledge of Specific Exam Content 


Before selecting any course of study, it is important to identify the instructor’s 
qualifications. Some instructor’s are unfamiliar with the exact contents of the P.E. exam or offer 
only selected areas of instruction. All programs and vendors that produced a comprehensive, 
self-sustaining P.E. refresher course were consider for the evaluation. Additionally, a program 


that was sponsored by a key engineering organization or a recognized educational institute were 
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considered more favorable than non-sponsored programs due to the likelihood of the 


organization being more familiar with the current P.E. exam content and format. This factor was 


assigned a maximum weight factor of 15 out of 100. 
5.6 Cost 


Of the courses identified, the costs ranged from $200 for a single instructional area to 
$2500 for an on-line, interactive, comprehensive refresher course. The cost was a factor in 
selection based on the products marketing potential. The content of the evaluated product could 
be compared to the overall cost to determine proper pricing for a similar course. The cost factor 


was assigned a maximum weight factor of 10 out of 100. 
5.7 Miscellaneous Factors 


Appendix C summarizes the critical factors and weight factors used for selecting the 
evaluated course. Aside from the previously discussed factors, several other non-critical 
selection factors were identified for this study. The number of lessons provided was considered 
to be less important than the total time required to complete the course; however a sufficient 
number of lessons were required to fully cover the major topic areas. This caused the course 
duration to be weighted 10 out of 100 points. Finally, the retake guarantee would typically be a 
critical selection factor if cost was more highly rated. The ability to retake the course was not 


weighted heavily (10 out of 100). 
5.8 Selected Course 

The course offered by Keepsmart Engineering, in coordination with ASCE, was selected 
to be the most applicable to this study and received a combined score of 86 out of 100. The 


selected course combines both a CD-ROM and Internet format and is designed to be a 7 module, 


50-hour self-paced program covering each major Civil Engineering topic. 
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After purchasing the course, the evaluator was provided with immediate access to an 
online version of the course. The course software was then shipped overnight to the evaluator. 
The CD-ROM combines both an audio and video instructional presentation, whereas the online 
version contains only audio clips due to the large bandwidth required for video. In both 
instances, help is available for each module through a central point of contact at Keepsmart 
Engineering via email. A database of questions is maintained at Keepsmart Engineering, and 
any new questions are forwarded to the instructor for evaluation. Questions are then answered 
via email back to the individual. Feedback time for a common question is between 30 minutes 
and 24 hours. In the event the student requests to discuss the topic or question with the 
instructor, Keepsmart Engineering contacts the instructor to obtain approval for one-on-one 
contact. However, since the instructors are full time engineers, availability to talk with them 


personally is dependent on their schedule. 
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6. COURSE EXAMINATION 


6.1 General Course Overview 


The selected course was divided into 7 different modules that cover the major topic areas 
in Civil Engineering. Table 2 lists all the major topic areas covered by the course. Each module 
consisted of four sections: a pre-exam, a demonstration problem, a practice problem, and a post- 
exam. The course moves through each section of the module, although the student has the option 


- to go directly to any specific module. 


Table 2: Course Content Summary 


Internal Rate of Effective 


Cont 


Present Cost 


Project w/ 


Perpetual Life 
Soil 
Classification 
Strength & 


Bearing Capacity 


Hydrostatics 


Rainfall Runoff 


Steel Beams & 
Trusses 


Water Demand 


Trickling Filters 


Traffic Volumes 
Timing & 
Signalized 

Intersections 


Circular & 
Vertical Curves 


Taxes Alternative Projects 
Effective Stress & 
Relationships Stress Distribution 
Structures 


Consolidation 


Piles Seepage 


Closed Circuits Friction Losses | Minor Form 
Losses 
: Manning 
Momentum Specific Energy ioe 


Concrete Arches & Concrete 
Beams/Columns 
Columns beams 
: Coagulation & Ground Water 
Stream ae ae 
Sludge Digestion Land Filling 
Traffic Signal Freeway Freeway 
Warrants Interchanges Analysis 
Trip 


Accident 


: Generation & 
Reconstruction 


Queuing 


Spiral Curves Grade Separation 
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Vehicle Speeds 


Prior to beginning each module, the evaluator completed an on-line pre-exam with the 
use of only reference 8. The pre-exam was only accessible through the online webpage and was 
printed through the use of the “print screen” option. The evaluator’s pre-exam answers were 
entered into the Keepsmart Engineering database and used to establish a subject matter baseline 


for the evaluator and ultimately measure the course effectiveness for that student. 


After completing the pre-exam, each module presented a demonstration problem 
focusing on the specific area. Typically these specific topic areas were similar to the questions 
on the pre-exam. Each demonstration problems was viewed and printed through Adobe Acrobat 
Reader. The solutions for each module were printed as a whole with the “printable solutions” 
option, or could be printed individually from the specific subject demo problem screen. Each 
problem consisted of several questions, each of which were multiple choice, and contained 


39 66 


embedded hypertext, for example “manometer”, “atmospheric pressure” and “head” as shown in 


Figure 1, that provided the user with definitions and equations for key words within the problem. 


Fj] SITUATION: 1. Hydrostatics Demo Problem 
Arectangular panel, 4 feet wide ‘anc: 6 feet lorig.{s held in a’tank at a depth of 9 feet, paralle!to the surface, An agen 
air U-shaped manometer is tapped’ at the bottom of the:tank. 


REQUIREMENT 
(d) What is the atmospheric pressure in terms of "head® of mercury? (Answer in inches:) 


‘Hiy26a (ily 248 


MEM 
MEN 


Figure 1: Example Demonstration Problem Screen 
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During the demonstration problem, the evaluator had the option to activate an instructor 
recording to have the “instructor” explain the basic concept behind the problem, the problem 
itself, or a specific problem requirement / question. In the CD-ROM course, a small screen 
appeared with a video clip of the instructor explaining the requested information. The online 


course only provided audio of the instructor’s voice. 


The third part of the module was the practice problem. Each module contained one 
practice problem for each major topic area. Each problem contained multiple questions with 
each successive questions depending on the previous one. The practice problem was basically a 
duplication of the demonstration problem, but with the numerical values changed as shown by 
comparing Figures 1 & 2. There was no instruction provided to complete the practice problem. 
The instructor non-quantitatively indicated, via audio, whether the evaluator had selected the 
correct or incorrect answer. The evaluator was continued selecting answers until the correct 


answer was chosen. 


q SITUATION: 1. Hydrostatics Practi¢e Problem 
A rectangular panel,-3 feet wide and 6 feet long Is held In’a tank at'a depth of 7 feet, parallel to the surface. An open- - 
air U-shaped manometer is tapped:at thé bottorn of the tank. a 


REQUIREMENT 
(4) What is the atmospheric pressure in terms of "head" of water? (Answer in feet.) 


(ipods (iv) das 


Figure 2: Example Practice Problem Screen 
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The post-exam followed the same format as the pre-exam and contained the same 


problem structure as shown in the demonstration and practice problems. The difference with the 
post-exam was that only one question was asked from each different section within the module, 


however this question was identical conceptually to the practice and demonstration problems. 


After completing the online post-exam, the questions were corrected within seconds and 
marked as correct (with a green checkmark) or incorrect (with a red x). The final results 
included the percent correct on the post-exam and percent improvement from the pre-exam to the 
post-exam. Correct answers to incorrect problems were not provided. An email was sent to the 
account on file stating the evaluator had successfully completed the given section with the 


indicated percentage correct. 


The evaluated course was heavily problem based with supporting theory added as 
necessary. The course was designed for the typical engineer preparing for the P.E. and geared 
towards individuals with a technical background, ideally with an undergraduate engineering 
degree. Although non-technical individuals could successfully complete the course, it would 
probably require the use of additional resources outside the individual refresher program. The 


program course was designed as a review course and not intended to provide initial instruction 
6.2 Screen Layout 

Each module contained a basic problem screen and was separated into three sections: 
problem statement, visual aids, and control menu. Figures 1 & 2 show an example problem 
screen with the problem statement in the top window, the visual aids on the lower right, and the 
control menu on the lower left. 


6.2.1 Problem Statement 


The problem statement screen listed the general situation, the solution requirement, four 


multiple-choice answers, and four command options. Within the problem requirements were 
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hypertext links that displayed the requested information in the visual aids section. The command 
options that were provided allowed the user to activate the instructor video recording (or audio in 
the online version), view additional visual aids that were related to the specific module, display 


the Greek Alphabet, or continue onto the next problem. 
6.2.2 Visual Aids 


The purpose of this screen was to display supporting information for the problem. This 
information included basic equations relating to the subject topic, associated tables and graphs, 
and video clips of tests that were performed to determine certain parameters within the problem 
(i.e. how to conduct a plasticity test). Additionally, when the problem was first introduced, the 
visual aids screen displayed a free body diagram or a sketch of the problem situation as shown in 


Figure 1. 
6.2.3 Control Menu 


The contro! menu provided the user with a means to navigate through the entire program, 
and access additional information about the resources used to develop the specific topic area. A 
list of resources and comprehensive glossary was accessed through the “tools” option. 
Additionally, one available tool was an internal scientific calculator, which was capable of 
performing very basic operations. Another key option was the control of the video instructor. 
The evaluator could start, stop, and replay the video clips; however, once the video clip began it 
could not be partially rewound. This caused the evaluator to replay the entire video clip even if 


only part of the instruction was required. 
6.3 Individual Module Evaluation Ratings 

The comparison charts in Appendix B-1 and B-2 summarize the ratings assigned to each 
individual module and is divided into delivery and content related factors. Table 3 summarizes 


the combinations of all 7 modules weight factors for a specific rating factor and shows the 


average value of each factor for the entire course. The entire course received a rating of 76.6%. 
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The factors associated with the content of the course module are shown in italics, whereas 


delivery related factors are in regular text. 


Table 3: Civil Engineering Review Course Evaluation 


oy ; > . 


Feedback is analytical, suggestive, and timely (not just provisions of 
he right answers) 


Ability to see and hear the instructor 


Active participation by the learner vice passively listening to lectures 
ritten text provided that supplements course lectures 100% 


yperlinks provided for more topic information 100% 
internal emphasis focusing on how instructional material relates to 


Course provides challenging, but attainable learning tasks 
Problem presents more information than necessary to allow student 
decide which information is relevant 


Course material presented in a manner that increases students self 

‘ ee ; : 3 100% 100% 
confidence (start with simple concepts before introducing new ones) 
Problems require students to make logical assumptions 100% 100% 
Problems require students to think critically and analyze information 100% 100% 


Student provided with a "hands-on" model 100% 


Student discussion of relevant P.E. experiences 2 100% 0% 


generat ae may ssi emcee 


hat are interesting a to the 


student 
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The majority of the factors listed in Appendix B concentrate on the delivery method used 
to present the material. Several factors concentrate on the individual content for each module. 
However, for the delivery method factors in plain text, the ratings are relatively consistent across 
all seven modules since the types of visuals, tools, resources, and screen layout were the same 
for each module. The only major change in presentation style was the individual instructor’s 
ability to accurately and effectively convey the topic to the student. The individual content 


factors will be discussed in section 6.4.3 through 6.4.9. 


Referring to Appendix B-1 and B-2, a hypothetical “ideal” classroom was designed to be 
used as a baseline for comparison with the evaluated course. The “ideal” factors were assigned a 
value of 100%. In many instances, key factors for a distance-learning, self-paced class 
(specifically this CD-ROM training course) were assigned a score of 0% due to the manner in 
which these factors were selected. For example, some of the factors that received an 
effectiveness rating of 0% included the ability to promote group learning, the use of a “hands- 
on” model, and the opportunity to share personal experiences with similar problems. Although 
all the factors were considered in rating the course, they are more applicable to a “new 


instruction” course and not directly applicable to a distance learning “review” type course. 


The ability to have face-to-face contact between students and instructors was assigned a 
value of 50%. In an “ideal” classroom, the student is able to ask questions of the instructor and 
receive immediate feedback. In this course, the evaluator was able to “see” the instructor in the 


video presentations, which made it easier to follow the instruction. 


During the course, the evaluator had the option to be actively involved in the instructions 
by working practice problems along with the instruction from the demonstration problem. 
However, the evaluator had the option to passively listen to the instructor and not actively 
participate. As a result, the active participation in class factor for this course was assigned a 
constant value of 50% for each module since the evaluator was not required to participate during 


the instruction. There were no outside homework assignments required to complete the course. 
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Finally, the course is designed to be self-paced which eliminates the ability to verbally 
express ideas and discuss problems with peers and coworkers. However the factor was assigned 
a rating of 20% since the typical engineer studying for the P.E. has the ability to discuss the 


problems with other engineers in the work place. 


6.4 Course and Module Impressions 


Overall the course appears to be an excellent tool in preparing for the Civil Professional 
Engineer exam. The entire course was presented in a very professional manner, with the proper 
emphasis given to each of the seven major topic areas. Each module contained an animated 
introduction slide, which provided a good initial impression of the quality and time involved in 
developing each module. The cost for the course was reasonable for the amount and quality of 
instructional content included. Based on the previously identified selection criteria, this course 


met all the selection criteria, and after evaluation, met the expectations of the evaluator. 


6.4.1 Overall Advantages 


The layout of the course was very user friendly. The main problem screens were easy to 
navigate through and provided all the necessary information to complete each problem through 
the use of hyperlinks and the available tools; therefore no additional reference material was 
needed to successfully complete the problems. All required tables and figures were included. 


within the program itself. 


The course instruction was presented in a clear and concise manner, which made it easy 
to review the topic even with the absence of the physical instructor. Problems were presented in 
a logical manner, and few questions arose that were not covered by another section of the 
module. In a few instances, the evaluator identified questions from unclear concepts during the 
theory discussion, but no questions arose as to the intent of the review problems or how to go 
about solving the problem. When a clarification was needed, answers were obtained from either 


Keepsmart Engineering or the instructor within 48 hours. 
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The ability to print out the solution to each demonstration problem was a great reference 
to be used during the P.E. exam. Additionally, the internal, scientific calculator provided a good 
resource and tool in calculating problems and allows the program to be a self-sufficient training 
course. The calculator contained specific help options that identified and defined each key. A 
final benefit of this course was the email message that indicated the evaluator’s completion of a 
specific module. This email provided existing registered Professional Engineers with credit for 


annual continuing education requirements. 


6.4.2 Overall Disadvantages 


To assist the IES in creating a separate, competitive program, several areas were 


identified in the evaluated course that would improve the course and add greater benefit future 
students. These areas were identified based on the ideal learning environment factors discussed 


in section 5.3. 


First, as identified in course evaluation factors, the “ideal” learning environment would 
include a text associated with the course material. A printable solutions option for the 
demonstration problems was available with this program through the use of Adobe Acrobat(C). 


The problem was with printing a demonstration or practice problem to be used as a worksheet for 


completing the problem. In this case, the page could not be formatted prior to printing and the 
problem printed on a single page. As a result, problems with multiple questions were printed in 
a very small font size. This made it difficult to read and did not provide sufficient room to work 
out the individual questions. As an improvement, the problem should be able to be formatted as 
necessary to increase the font size and provide sufficient room to complete all question 


calculations. 


One measure of an ideal classroom is whether the concepts are presented in a variety of 
contexts and expressed in a variety of ways. The problems presented in this course were 
repetitive. As previously mentioned, all four problems (pre-exam, demonstration, practice, and 
post-exam) were formatted in exactly the same manner with the exception of the actual values 


within the problem. The course teaches how to complete a specific type of problem and did not 
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expose the evaluator to other methods of presenting the problem. Additional practice problems 
should be added as a supplement to the course, which covers the same topics already presented, 


but provides the evaluator with the ability to see a problem presented in a different manner. 


As indicated in Table 3, in order to be effective, feedback during a course needs to be 
analytical, suggestive, and timely. Typically, the response to an incorrect answer was that it was 
wrong and did not indicate why it was wrong. Similarly, correct answers were not quantified, 
only indicated as correct. For example, in an economics question involving the comparison of 
two or more companies, the question asked which company had the highest rate of return. The 
answers were company A or B. When the correct answer was selected, the instructor responded 
with an audio clip like, “Correct, Company A has the higher rate of return.” The actual 
calculated rate of return should be provided to serve as confirmation that the problem was 
calculated correctly and not just arrived at by chance. The audio clip should respond like, 
“Correct, the ROI for Company A is 18%.” Similarly, if the answer is incorrect, the audio clip 


should explain why that answer is not the correct answer. 


In order to understand the topics being presented, each term needs to be clearly defined, 
or the ability to define the term should be made available to the student. This course provides an 
extensive technical library that is accessed via hyperlinks throughout the problem text. Although 
the definitions listed are very helpful, there is no search engine included that allows the student 
to look-up specific terms. As a convenience to the student, the ability to enter a technical term 


and obtain its definition is not necessary but would be an added benefit to the program. 


Finally, once the video instructor is activated, the evaluator was required to listen to the 
entire pre-recorded lecture segment, which could last between 10 and 60 seconds. There was not 
an option to partially rewind or fast-forward the instruction. This proved to be an inconvenience 


when only a section of the instruction was needed to clarify the problem for the student. 
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6.4.3 Economics 


Although there is not a specific economics section on the new breadth and depth Civil 
P.E. Exam, the concepts are important to understand and may appear within one of the five 
major Civil Engineering categories. The economics module is divided into 7 separate sections 
that cover the topic areas shown in Table 2. Fundamental concept coverage, feedback, how 
material relates to everyday reality, and concepts being presented in a variety of manners are all 


evaluation factors that address individual module content. 


The feedback for economics received a rating of 50% since many of the answers were not 
quantified by providing the student with the correct number. Additionally, the module did not 
explain why the other answers were are incorrect, which makes it difficult to determine where 


the error in the calculations occurred. 


How the economics problems related to reality received a rating of 50% primarily 
because each topic was presented as a textbook problem with little reference to how it applies to 
a real-life situation. Additionally, each topic was presented in one manner. There was no variety 
between demo and practice problems. Finally, each of the topics in the module presented only 
the information necessary to solve the problems which eliminated the need for the student to 


decipher the important values. Asa result, this area was given a value of 80%. 


6.4.4 Geotechnical 


The entire Geotechnical topic was covered very extensively and thoroughly. Under the 
Soil Classification section, there was a detailed video clip of how to perform a liquid and plastic 
limit tests. This video was informative but added little value to understanding how to complete 
the problems. Conversely, the video clip provided in the consolidation section was very 
beneficial in understanding the consolidation theory. It clearly defined the theory and 


demonstrated its application. 
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The individual values assigned to the Geotechnical evaluation factors were higher than 
the economics section. Compared to available reference material, the topic appeared to 
completely comprehensive and covered all major topic areas. Additionally, the feedback on 
correct and incorrect answers was more qualitative and provided the actual value corresponding 
to the answer which caused the feedback rating to be higher at 70%. This value was not assigned 
a 100% value since 100% represents real-time feedback from an instructor in a classroom 
setting. Finally, the problems contained information in excess of that necessary to complete the 


problem which caused this factor to be 100%. 


6.4.5 Hydrology and Hydraulics 


The module provided an extensive review of the topic. The sections discussing closed 
circuit, friction loss, minor losses, and momentum were very well integrated. However, the 
module needed to include more discussion on pumps and turbines, reservoir sizing, and 
Bernoulli’s equation. As a result, the comprehensiveness of the course only received an 80%. 
The rest of the individual evaluation scores were similar to the Geotechnical section with the 
exception of problem information. This factor received a value of 80% because no excess 


information was provided in the problem. 


6.4.6 Structures 


Structures is a very broad topic and difficult to cover all the specific aspects of structural 
design in a short refresher course. The module covered 5 major topics, but should have included 
several additional topics that are discussed in section 7.0. As a result, the comprehensiveness of 
the module received a rating of 70%. The other rating factors were identical to the Hydraulics 


and hydrology module. 


During the instruction of this module, the instructor discussed the process behind solving 
the problem rather than immediately discussing which formulas to use for the problem solution. 
This brief process introduction was very beneficial in understanding how to approach each 


problem. The drawback to this module was the lack of requirements for each problem. Many of 
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the demonstration problems only contained 3 requirements with only 1 of those requirements 


being asked in the practice problem. 


6.4.7 Environmental and Sanitation 


This module was very comprehensive and covered all the major topic areas completely 
which caused the comprehensiveness factor to be assigned a value of 90%. The ratings for this 
module were the same as economics except the comprehensiveness factor. The benefit of this 
module over the others was that it clearly quantified correct and incorrect answers so the 


evaluator could determine where the mistake was made in the calculations. 


6.4.8 Transportation 


Thorough coverage of all topic areas with the exception of specific roadway design | 
which resulted in a rating of 90% for the module comprehensiveness. Additionally, the feedback | 
given for each problem, as well as the remainder of the evaluation factor ratings, mirrored the 
Economics section in that it did not quantify “yes”, “no”, “true”, and “false” answers. If 
answered incorrectly, there was no explanation at why that was the incorrect answer. This 
module also contained several video clips that showed how traffic lights were timed, how lane 
sizing was accomplished, and how an Automatic Traffic Recorder functioned. Although 


informative, these video clips were not necessary to fully understand these concepts. 


6.4.9 Surveying 


Although there is not an afternoon section on the P.E. exam dedicated to surveying, the 
concepts in this module are incorporated into the 5 main topic areas. This entire module was 
extremely comprehensive and provided the best feedback of any of the other 6 modules. When 
an answer was incorrect, the instructor indicated where in the calculation the error had most 
likely occurred. Additionally, each correct and incorrect answer was quantified which resulted 
in the feedback factor being assigned a rating of 80%. The remaining evaluation factors received 


ratings identical to the Geotechnical module. 


26 


7. COST ANALYSIS 


7.1 General 


A break even analysis was completed to determine whether the creation of a competitive 
product will be cost beneficial to the IES. According to the NCEES, an estimated 12,000 
engineers take the Civil P.-E. Exam annually. This break even analysis assumes a linear sales 
growth to achieve 10% and 20% market shares in the U.S. market for distance learning products 


of this type. 


7.2 Product Costs 


The cost to develop a new course includes both fixed and variable costs. Fixed costs 
include all costs that must be paid regardless of the amount of product sold. In this analysis, all 
the production and marketing costs were considered fixed costs. Variable costs are dependent on 
the number of units sold and are typically linear in nature. The combination of fixed and 
variable costs are added to obtain the total cost of the product. Table 4 summarizes the fixed and 


variable costs for this study. Most of the costs are self explanatory. 


A previous analysis suggests that the addition of an “on-demand” instructor would be an 
attractive service to students purchasing the IES product. For this, the cost of a dedicated help 
line was included at an assumed rate of $150/hour. Additionally, only 30 minutes of questions 
was estimated for every 10 units sold. This equated to $7.50 for product support per unit sold. 
The waste factor takes into account damage CDs and errors in reproducing both the text and CD 


information. 
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Table 4: Total IES Product Costs 


Quantit Unit Unit Price | Total Price 
hrs 
ES Instructor Development Hrs 200,000 
hrs 
hrs 


Saal 
| hrs | 
| Hrs | 
IES ProjectManagement__| 80__—*|_—ihrs_—| =~ $40 | $3,200 
ahs | 
—— 
a 


Fixed Costs: 


evelopment 
115,000 


$1,000 

best 

rarer) 

80 

24,000 
a eae 


otal Fixed Costs: | Es $333,200 


ariable Costs: 


hipping 1 
1 


M1O1O 


O 


7.3. Break Even Analysis 


There is evidence that engineers preparing to take the P.E. examination are moderately 
sensitive to course prices. Based on a survey conducted by IES, 20 out of 27 engineers would 
pay up to $500 for a refresher course. This analysis considered selling prices between $200 and 
$500. As shown in Figure 3, if the selling price was $300, the IES would need to sell 
approximately 1350 units before making a profit. Similarly, at $400, the IES would only need to 


sell 950 units break even. 
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Figure 3: IES Break Even Analysis 


The target market for the product is nationwide with the potential to reach internationally. 
Data from NCEES indicates that in 2001, the number of North American engineers applying for 
the exam was approximately 12,000. This represented an increase of almost 1,000 applicants 


throughout the country from last year. This growth is expected to continue in the future. 


The JES product is likely to obtain a small share of the market shortly after launch. This 
share is likely to increase with time. Two assumptions are made in the market share growth. 
First, a pessimistic growth rate of 2% a year for 5 years will yield a total market share of 10% 


and would require 2.0 years before IES would make a profit as shown in Figure 4. The second 
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assumption is an optimistic growth rate of 4% a year for 5 years to obtain a total market share of 
20%. In this case, the break even point would occur at 2.8 years as shown in Figure 5. In both 
instances, a constant market of 12,000 engineers nationwide and a selling price of $300/unit was 


considered. 


—— $300 Selling Price 
——— Fixed Cost 
—*&— Total Cost 


($k) 


Years 


Figure 4: Break Even Time based on 2% Annual Growth 


—@— $300 Selling prce 
—— Fixed Cost 
—4— Total Costs 


Years 


Figure 5: Break Even Time based on 4% Annual Growth 
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7.4 Results 


Engineers preparing for the P.E. exam are only moderately price sensitive. Addtionally, 
if available P.E. distance learning courses are priced relatively high, the costs may be beyond the 
limit acceptable by engineers. Therefore, by maintaining the selling price of the product below 
the competitors, this new product will be more marketable. The current market price for a 
typical CD-ROM refresher course is approximately $500. By selling the product at $300 per 
unit, the expected sales growth will be greater than selling for a higher price. Additionally, a 


profit will be made from the product around 4 years after initial introduction. 
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8. CONCLUSIONS 


Professional Engineer review courses can be effectively developed and delivered through 
means of a distance learning format. There is a market for a distance learning P.E. review 
course. Engineers preparing for the exam will consider purchasing a distance learning review 
class course. A survey conducted by the IES indicated that 88% of engineers in the State of 
North Carolina would purchase a distance learning refresher course. Of these engineers, 82% 


would purchase a combined CD-ROM and textbook refresher course. 


Several distance learning options are currently available. The on-line/offline combined 
CD-ROM course developed by ASCE and marketed by Keepsmart Engineering was an excellent 


program and proved to be an adequate benchmark for IES for future course development. 


Profession Engineer examinees are price sensitive and cost is an issue to be considered in 
course development. Most engineers would only pay up to $500 for a quality, comprehensive 


refresher course. 


The overall evaluated course provided a very effective tool in reviewing for the P.E. 
exam and is recommended for any Civil Engineer preparing for the licensure exam. The course 
was very “user-friendly” and easily navigable. The screen layout and the ability to have the 
instructor explain how to complete the individual problem requirements were two benefits of this 


course. 
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9. RECOMMENDATIONS 


A review of the ASCE refresher course content revealed several concepts and items that 


should be included and considered in developing a new course, such as: 


e Ease of navigating through the course 

e Comprehensive topical coverage 

e Video and audio clips of instructors explaining the topic 

e Explanation of the process involved in solving a problem 

e Effective use of visual aides and graphics to fully explain the problem 
(i.e. Freebody diagrams) 

e A diverse variety of problems 

e Strong imbedded scientific calculator 

e Quantitative feedback on problems 


e Dedicated help line to answer basic questions 


The cost analysis in section 7 included variable costs for required student help. To 
minimize this cost, it is recommended that an online bulletin board be established as the primary 
means of answering and asking questions. This would allow students to share solutions to 
practice problems and greatly reduce the direct interaction time of the instructor or IES 


personnel. 


The designed length of the evaluated course was sufficient for the information given. 
However, the course could be lengthened without reducing the desirability of the course, by 
adding additional practice problems. Each individual module took between 4 and 10 hours to 
complete. This time was also determined to be adequate for reviewing a given section. Overall 
a new course should be designed to be completed in less than 100 hours, provided additional 


practice problems and the subject areas discussed below are included. 
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The evaluated course contained several video clips that displayed tests or demonstrated 
the theory behind the problem. Although graphics and diagrams are important in an Engineering 
class, these items can be removed from the review course. For example, video of how water 
flows through a pipe requires time to develop the graphic, but adds little to understanding the 


concept. Free-body diagrams that are not animated should still be used to explain the problem. 


When explaining how to complete a problem, the process should be discussed prior to 
beginning the solution, similar to the structures module. This will help the student understand 
how to go about solving the problem and give the student an idea of how much work is involved 


to obtain the correct solution. 


A new course should consist of the same 7 major topic areas that were covered in the 
evaluated course. In addition to the subject already covered under each module and as listed in 
Table 1, the subject matters listed in Table 5 should be added or discussed more in-depth. These 


items were identified in Reference 8, but were not directly covered in the refresher course. 


The format used in presenting the material needs to be user friendly and easy to navigate. 
The example screens shown in Figures 1 and 2 are a good baseline for design. At any given time 


through the problem discussion, the student should be able to view the free-body diagram and the 


problem statement. 


The tools section of the evaluated program was very useful. Specifically, the calculator 
allowed the program to stand-alone. However the imbedded calculator should be redesigned to 
handle multiple calculations at one time. As a recommendation, the calculator should be 
modified to provide a “( )” or similar function to allow for long strings of calculations to be 
entered. This recommendation is based on personal preference with using calculators and is 


partially substituted by the inclusion of a memory addition, subtraction, and storing capability. 
As a follow-on to this project, and prior to designing a competitive product, a complete 


market analysis needs to be completed to identify the number of engineers throughout the 


country that have used refresher courses in the past and estimate the expected usage in the future. 
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Additionally, the expected selling price, revenue, and payback period should be identified to 
determine how effective a competitive course would be in the marketplace prior to beginning the 


development of a new course. 


' Table 5: Additional Subjects to be Discussed 


Add 
Economics Break Even Analysis 


More Discussion on Phase Diagrams 
Geotechnical Mohr’s Circle review 
Surcharge Loading 


Review of Bernoulii’s and Energy 
Equations 


nats ins Pumps and Turbine Efficiencies and 
ee Horsepower 


Reservoir Yield and Sizing 


Stress / Strain Relationship Review 
Non-Cantilever Beams 


Indeterminate Structures 


Structures 


Pre-Stressed / Post Tensioned 
Different Connections (Welded, Bolted 
More Discussion on Water Chemistry 
Calculations 
Pavement Desig 
Parking Desig 


Sanitary Sewer and 
Environmental 


Transportation 


No Additions Recommended 


An additional recommendation to justify the ratings assigned to each “learning” 
environment factor in the evaluated course, a separate survey should be performed on individuals 
that have participated in distance learning classes to determine the effectiveness of each factor. 
This survey should be completed prior to developing a new course to determine the course’s 


marketability. 


a5 


10. REFERENCES 


. National Council for Examination of Engineers and Surveyors (NCEES) (1998), 


Organizational Home Page. In ncees.org [On-line]. Available: www.ncees.org. 


. American Society of Civil Engineers (ASCE) (2001), Organization Home Page. In asce.org 


[On-line]. Available: http:/Awww.asce.org/distancelearning/pe_exam.cfm. 


. National Society of Professional Engineers (NSPE) (2001), Organizational Home Page. In 


nspe.org [On-line]. Available: http:/Awww.nspe.org/home.asp. 


. Science for All Americans Online (2000). “Chapter 13: Effective Learning and Teaching.” 


In: project2061.org [On line]. Available: www.project2061.org/tools/sfaaol/Chap13.htm. 


. University of Idaho Engineering Outreach (2000). Distance Education at a Glance. In: 


uidaho.edu [On line]. Available: www.uidaho.edu/evo/distglan.html. 


. The Adult Learner in Higher Education and the Workplace (2000). Seven Characteristics of 


Highly Effective Adult Learning Programs. In: newhorizons.org [On line}. Available: 
www.newhorizons.org/article_billington| html. 


. Prasuhn, Alan (1995). “Early History of Professional Registration in the United States.” 


Journal of Professional Issues in Engineering Education and Practice, ASCE, vol 121, pp 
25-29. 


. Lindeburg, Michael R. (1997). Civil Engineering Reference Manual for the PE Exam (o" 


Edition). California: Professional Publications, Inc. 


36 


Ape nsou sor wm ; ade] oapiA 


ade] oapip 
7ASoajuUeluedap/npe Uingne bua MM 


Bio‘easeMuouses| Bio UOUIeS; MMM ede! oapip 


woo’ syooqd|b@Ajes}sno ° : ede] oapi, 


woo’ joemesnibu Bid IOUT MMM 1X9] pede jies 


LOD JOJDBAPSI@ s}soyudnosHoa\| 6S IOJDSAPST MAM 


UOHEWIOJU] 
npaesmn ules|@Bbue| }XOL psoed jJeS 
/SHOeIED/SiUSpniS/Npe UlSUODSIM UleS| 


you yul| yee@outosed (W090 DUIDIEd MMM 1X9, paokd Jes 


you'sAsus@yesie} you sAsUa MMM JOUI9}U] SAT] 


auozbulssedauy/wo0o ssedz! 


WOO! JeYD 
/ Pleogy|eByD suljuO 


npa ynossiu@[uosupje Npo LNOSsIur gam 


Bio‘adsu@Asesjsno 


woo" Pewsdee7@)ueboAeuelp 


oulT-UO 
Wwew3 90d 
poujaw 


SHONWHAATa ALISHAM ASUMNOD *V XIGNAddV 


Poe cee r-t~—~S@YS c. —T-lllcel Oe—axXlC“C'' Eder dg 


AWSJBAIU 


AJEIS CUO YON 


SS3Jq S9y7e7 }eBID 


IDW 


UISUODS! AA JO ASJBAIUQ 


@uo7 Bulsseg ou 


WNossiy] 


AdSN 


yews dsay 


uoneziuebio Bulmer 


shem jo AJQUeA B Ul passaidxa pue $}xa}U09) 
fO AJBUeA B Ul pajUuesaid ale s}]dsqu07 
yIeod 

AepAiane 0} Sajejai peiajeus euowonujsul 
Moy Uo Bulsndoj siseydula jeusdju] 
UONBWIOJUI 

d1do} SOW JO} pepiAoid sequesEjoy 
Sainjo9| 

@Sinoo sjuswajddns yeu} paplAoid 3x9} US} 
$91N}99] 0} Bulu9s}si] 

Ajanissed SA Joue9| ay} Aq uoedionied aaioy 


%O0O0b %0O0b %001 %00+ v 


. (siomsue jy6u ay} Jo suorsjaoid jsnf jou) 
708 Ajeuy pue ‘eAnsebbns ‘eoAjeue si yoeqpeey 


%O00L 
%00t 


%00L %00L %00L %00L %001 %00L g 
%06 |  %06 | %00L %00L %001 g 
|  %00L | %OZL %06 | %001 %06 g 


ee 


%001 %001 %001 %001 %001 %001 %001 | 9 | 


SAISUBYSIAUIOD S} JU@JUOD 9SinoyD 


Ei 


(aayew poolqns oO} ajqeodde ‘a;qibo| 
%001 %001 %001 %001 %00} %001 %001 ‘QNJBUUOJUl) Sple |ENSIA Jo esn eANoaN3 
%00L %OO0L %00L %00L %00L %001 %O00L 
%001 %001 %001 %00t | 9 | 
sJayoea’ 
Be 3 JPA gf Pee 
MOMS |< = yealUYyoO}ONS) |SoIUIOUOD 


SONDINVa GNV SYO.LOVA NOILVATVAd ASANO) ONINAVAT GONVISIG AYVW Tad *1-& XIGNAddVv 


‘s1ojoRy , AIOATOP,, 


%IL ALL 


ca 


yUssoidal SMOI IOY}O [TY *S10}0BJ ,.JUD]U0,, yUasoIdol SOIR] Ul SMOY :O}0N 


WSL GL 


%001 jejo, ASAeg / Ws0- 


uapnis au} O} Jel|Wey pue Buseojul 


eje yey] suolsenb sasod uononpowju| 


seoueradxal 
"Fd JUueASISJ JO UOISSNOsIp JuSpN}s| 


%001 %001 %001 
%001 %001 %001 


%001 


%001 %001 
xe |e 


uojeuojul azAjeue pue 

Ayjeoniso yuly} 0} s}USPN}s sesinbel Swajqold 
suoljduunsse 

jec1bo; aye 0} s}UapN}s sasinbe swa|qold 


MAU Bulonpowjul 8J0jaq s}deou0s ajdwis 
U}JIM He}s) BOUSPYUOD JJas s}JUapN}s SeseasoUl 


yeu} JOUUBLU e Ul peyUeseid jeayeU asuNnoD 


JUBAG{A/ S} UOEULIOJUI 
Yyolym aploep juapNys moje 0} Aiessaoeu 
UBY} UONCULIOJU] S1OU! S}UBSdId LWI9{QOld 
syse) Bujuied 

‘Bulbuayeyo sepiaoid asinoyD 
Buluses; dnoig ajowold 
$Jayoed} pue siovad 

UyIM Sle|qoid ssnosip pue seap! ssaidxe 
Ajjeqsaa 0} Aiqe ou} UM SjuapNys SEPIAO!d 


%00L 


%00t 
%0 


eqeuieye yng 


%0S 


SYO DIVA NOILVATVAd ASHNOD) «TWACGL, AUVINANATdd AS *0-& XIGNAddv 


T-O 


“SYJEUI IOYSTY PIAINI1 FdSN JO ‘STAON “AOSV Aq pososuods sasmog xx 
“ale OIj1d0ds 3eY} JO} USAIZ JUIOd Jo JUNO ay} syUasoIdal SoXOg [ENPIATPUI St} UI sIOqLUNN 


“A10893e9 Iofeut yoo IOF pamoy]e sjurod Jo JUNO UINUUTXELA SMOYS MOI 3Y} JO do} sy) ssoIDe JaqUINN , 


uouses] 


(g) (0) (St) AYSIBAIUL) AJE}S 
soueA suossd] p aseyoind 0} 000‘r$~ adel capi, euljoseD YWON 


usd 


(o1) (GL) Asusaiun 
40}ONIYSU O} [JEW ode, oapla uunqny 


(or) 0 () C SSald 
SoA VIN suosse} J sess sexe] Jea1g 
(OL) (g) (OL) (1) UISUODSIM 
JOJONIJSUY 0} [HUI wea, | SUOSS9| S} 0zr$ ~ jo Ayssaniun 
OL 


gajueieng yoeg Asuow 


(01) 


ayeJay se1y - SAA 


dea }xau SSejo awes 
5 [ul |}o1U2 0} 3aB No - sa, 


(OL) 


pepeeu se yonw 
se osn ‘WOY-GD > S8A 


0 8 woo'uozeuny 
SOLIeA suosso| 2 06$ 400g / NOU-GD 
(0) (0) é 


J0}FINAJSU WIN VIN SsJOquisu 
jeuosieg poubissy 40} G6"ESE$ JO SE'6LES 


(Gt) (¢) 
syUuoW p ~ paced 9S pseoqyjeyy eujuo| Buisseg eu 


}X9] P99 JOS 


“paeog o}IYM sUljUO 
ue 0} pajsod suonseny 


(02) 499M / SUOISSS (G1) 
OATT — SPA SINOY UBAVS ~ Z yousazU] BATT 
009 — 00S 
(Gb) Om 
“pseog 8}14yM eul|uO paseg-}2S W00d eUD | 


ue 0} pajsod suonseny eUMONL’ SUOSSQj 8 paeoqyjeyy ouluo 


(9) 
julod jeajues (9) (4dSN 
0} suonsenb jews Sanoy OS fasaueyy (02) 9 3OSV JM) 


G6, | uonnfosvulsiaoda suoseaps 3OSV 403 S6p$) —_feulUG 40 WON-GO 


yews daay 
JO O@piA UOROW |IN4 aINpow / PZ$ 40 GESS 


s810}9e4 1YBIa AA 


SHANLVad LoOAdOdd AO NOSRIVdWNOD *0 XIGNaddV 


